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METHOD OF ORGANIZING DIGITAL IMAGES 
ON A PAGE 
FIELD OF THE INVENTION 

The present invention relates to a method, system and computer 
5 software program for automatically organizing digital on a page that is especially 
useful when images of various sizes are presented for placement on the page. 
BACKGROUND OF THE INVENTION 
Photographic albums provide a way to store, organize, and display 
pictorial information. Typically this pictorial information corresponds to 

10 important life memories of the individual who created the photographic album. 
These memories can correspond to photographs, and keepsakes and mementos 
such as, greeting cards, invitations, hand drawn images, etc. Each year an 
extremely large number of photographs are developed, looked at and stuffed in a 
drawer or shoebox. Digital images suffer the same fate as their hardcopy cousins 

15 except in this case they are "stuffed onto" digital storage devices such as hard 
drives and CD-ROMS. The individual usually has good intentions of creating a 
photo album, but never quite gets around to it. This is because the process of 
creating a photo album is a time consuming, difficult deed that usually provides 
less than satisfactory results. 

20 Typically the process of creating a photo album is done by either 

cutting and attaching hardcopy images onto an album page or by scaling, 
cropping, and inserting digital images into album pages via commercially 
available imaging software such as Microsoft Picturelt™. One important step in 
creating an album page is the ability to easily arrange and fit the desired number 

25 of digital images on the album page. One way of simplifying the process of 

arranging digital images is to use templates that have predefined locations for the 
images. Templates provide an easy way to album images but they are not always 
efficient in their use of space. With templates the selected images are either 
proportionately sized or cropped and sized to fit in the selected location in the 

30 template. Proportionately sizing the image to fit a template can lead to 

considerable space being wasted on the album page. The process of cropping an 
image to fit a template can lead to important pictorial information being cut out of 
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the image. A further disadvantage to templates is the user can only choose from 
the available templates. If there are no acceptable templates available, the user is 
forced to resort to the tedious process of individually scaling, cropping, and 
placing the images on the page until an acceptable arrangement is found. With the 
5 advent of Advanced Photo System, digital scanners, digital cameras, and digital 
image editing software, digital images can have a multitude of various aspect 
ratios. This can make the process of arranging images on a page even more 
challenging. In addition, it is impossible to have templates that can facilitate all 
the possible combination of images. 

10 The present invention solves many of the problems of the prior art. 

It provides an easy and efficient way of arranging digital images of various size 
aspect ratios on an album page. In addition, the current invention provides the 
user with an easy way to generate a large number of different arrangements of the 
same images and thus allowing the user to choose a preferred arrangement. 

15 SUMMARY OF THE INVENTION 

The above, and other objects, advantages and novel features of the 
present invention will become more apparent from the accompanying detailed 
description thereof when considered in conjunction with the following drawings. 

In accordance with one aspect of the present invention there is 

20 provided a method of organizing a plurality of images in a predetermined page 
format, comprising the steps of: 

grouping the plurality of images into a plurality of different page 

layouts; 

analyzing each of the different page layouts in accordance with a 
25 predetermined criteria; and 

selecting the page layout based on the predetermined criteria. 

In accordance with another aspect of the present invention there is 
provided a system for organizing a plurality of images in a predetermined format, 
comprising: 

30 a first computer for composing a plurality of digital images on a 

page; 



a software program such that when loaded on the computer will 
cause the computer to group the plurality of images into a plurality of different 
page layouts; 

analyzing each of the different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on the predetermined criteria. 

In accordance with another aspect of the present invention there is 
provided a computer software product for organizing a plurality of images in a 
predetermined format comprising a computer readable storage medium having a 
computer program which when loaded into a computer causes the computer to 
perform the following steps: 

grouping the plurality of images into a plurality of different page 

layouts; 

analyzing each of the different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on the predetermined criteria. 

In accordance with yet another aspect of the present invention there 
is provided a method of organizing a plurality of images in a predetermined page 
format, comprising the steps of: 

providing a plurality of digital images; 

selecting a number of the images for placement on the 
predetermined format; 

grouping the plurality of images into a plurality of different page 

layouts; 

analyzing each of the different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on the predetermined criteria. 
BRIEF DESCRIPTION OF THE DRAWINGS 

In the detailed description of the preferred embodiments of the 
invention presented below, reference is made to the accompanying drawings in 
which: 
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Fig. 1 is a schematic diagram of a system for practicing the present 

invention; 

Fig. 2 is a page having a template layout in accordance with the 

prior art; 

5 Fig. 3 is plan view of a plurality of images for placement on an 

page; 

Fig. 4 is a plan view of the page of Fig.2 having the images of Fig. 
3 placed thereon; 

Fig. 5 flow chart of a method for determining a page layout for a 
10 plurality of images in accordance with the present invention; 

Fig. 6 is plan view of the format of a page on which the images of 
Fig. 3 are to he placed in accordance with the present invention; 

Fig. 7 is a flow chart illustrating one of the steps in calculating the 
page layout method of Fig. 5; 
15 Fig. 8 is an initial image page layout of the images to be placed on 

the page; 

Fig. 9 is a modified image page layout of Fig. 8 after further 

manipulation; 

Fig. 10 is a modified image page layout of Fig. 9 after yet still 
20 further manipulation; 

Figs. 11-15 illustrate yet further manipulation that may be made to 
the image page layout; and 

Fig. 16 is flow chart of a method of providing further image page 

manipulation. 

25 DETAILED DESCRIPTION OF THE INVENTION 

In the present invention, the term page as used herein is meant to 
include an album or scrapbook page, poster, soft copy display, or any other format 
where images are displayed. 

FIG. 1 illustrates a system 10 that can be useful in practicing the 
30 present invention. The system 10 includes a personal computer PC 12 containing 
a central processing unit (CPU) that can execute a set of predefined steps in 
carrying out the method of the present invention. A digital storage media 20 is 
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also provided in PC 12 for storing digital images. The digital storage media 20 
can include different types of devices, such as RAM, ROM, hard and floppy 
drives, etc. The digital storage media 20 can also be used to store the generated 
image page. In addition, digital image capture devices such as scanner 28 and 
5 digital cameras 30, which are additional sources of digital images, can also be 
provided to the computer 12. It is to be understood that the digital images may be 
obtained from any source. A user interacts with the computer 12 via input devices 
40, such as a mouse and/or keyboard, and a display monitor 50 that is connected 
to the computer 12. The system 10 may also contain a device such as a printer 61 

10 for outputting the image page. Alternatively, the above components do not have 
to all reside on the host computer 12 but can reside on a server 63 located at a 
network service provider 64 which can be connected via a communication 
network 70. The communication network 70 may comprise the Internet 74 which 
can be accessed by an individual using an Internet Service Provider (ISP) 76. The 

15 network service provider 64 may also include a customer database 66 for storing 
information regarding customers and a image storage data base for storing of 
digital images provided by the customer either by the internet 74 or by any other 
means. The service provider 64 will be equipped to provide goods and/or services 
as described herein or any other goods and/or services desired. The remote 

20 network service provider may also be accessed by a customer using a retail kiosk 
or any other appropriate communication device. 

Referring to Fig. 2 there is illustrated a prior art, a page 41 that 
incorporates a template 42 made in accordance with the prior art. The template 42 
comprises a plurality of predefined locations 43 - 47 for the placement of images 

25 that are used to simplify the process of arranging images on a page. A weakness 
of using pages with a predefined template becomes very apparent when the 
images selected to be arranged on the page have very diverse aspect ratios. FIG. 3 
shows five images 54, 56, 58, 60, and 62 that are to be arranged on a page and 
FIG. 3 shows a page that has predefined locations for the five images. FIG. 4 

30 shows the results of proportionately sizing the images to fit in template 42. As 
can be seen, this leads to a considerable amount of space on the page being 
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squandered. That is, large areas of the page are void of images. In addition the 
size of the images are substantially reduced when they are sized to fit the location. 

FIG. 5 is a schematic flow chart illustrating the method of 
constructing an image page layout according to the present invention using 
5 computer software program made in accordance with the present invention. A 
plurality of digital images (such as shown by FIG. 3) that can be placed on an 
image page are stored in a database at step 100. In the present invention, digital 
images refers not only to images obtained from photographs, but to digital images 
obtained from any source, for example but not by way of limitation, digital 

10 camera, scanning of a hard copy document, or electronically from another source. 
The format of a page 80 (see FIG. 6) on which the images 54, 56, 58, 60, and 62 
are to be arranged is selected at step 110 by either specifying the height and width 
of the page or alternatively the aspect ratio (i.e. width/ height or height/width) of 
the page can be specified. The digital images to be arranged on the image page 

15 are then selected at step 120 either manually, semi-automatically, or automatically 
from the database where the images are stored. The semi-automatic and 
automatic selection process, can make use of data, such as time and date, that is 
recorded along with images and digital image processing techniques, such as 
image content analysis, to help select relevant images that should appear on a 

20 given image page. Optionally, the images 54, 56, 58, 60, and 62 to be arranged on 
the page can be normalized at step 130. Normalizing the images prevents one 
image from spatially dominating the page layout. This is especially true when one 
image is much larger or smaller than the rest of the images. A preferred 
normalization is one in which the images are isotropically scaled so that their 

25 shortest dimension (height or width) are all equivalent. The normalized height 
and width of the images to be arranged on the image page and the format of the 
image page are passed to the page layout subroutine 140. The layout subroutine 
140 calculates a page layout of the images on the image page 80 and displays 150 
the results on display monitor 50. At this point the user can either accept 160 the 

30 image page layout or iterate through the page layout subroutine 140 until an 
acceptable image page is obtained. When an acceptable image page layout is 
obtained at step 170, the image page layout and images may be stored on the host 



computer 12, printed by printer 61, or transmitted to another site via the network 
70 where it can be stored, printed or view by another person. 

The job of the page layout subroutine 140 is to fit a given number n 
of images on a given image page to obtain a suitable page layout using a 
5 predetermined criteria. In the embodiment illustrated the predetermined criteria 
comprised fitting the n images onto the image page in such a way that minimizes 
the white space. The white space, which is the area of the image page not covered 
by an image, is defined by the following relationship: 

EQ. #1 White Space = 1.0 - (total image area)/(page area) 
10 Wherein: the total image area is the sum of the area of the 

individual images to be placed on the page; and 

the page area is the area as defined by the height H and Width W 
allowed for placement of the images. Generally, the page area would be the entire 
page. 

15 The page layout subroutine 140 may take into account the aesthetic 

considerations of the image page layout. One important aesthetic consideration is 
the spatial balance of the images on the page. Spatial balance is a measure of how 
equally distributed the images are on the image page. 

The problem of generating an acceptable image page layout that 

20 contains n images can be formulated as a combinatorial optimization problem. 
The most practical way of solving combinatorial optimization problems is to use 
stochastic algorithms, such as simulated annealing or genetic algorithms. These 
combinatorial optimization algorithms along with other optimization algorithms 
are described in Iterative Computer Algorithms with Applications in Engineering: 

25 Solving Combinatorial Optimization Problems by Salt, S. M. and Youssef, H., 
IEEE Computer Society, Los Alamitos, Calif, 1999. 

Referring to FIG. 7, there is shown a preferred embodiment of a 
flow chart for a page layout subroutine 140 for calculating the layout of the 
images on a page according to the present invention. The subroutine 140 in the 

30 embodiment illustrated is automatically executed by a computer software program 
that is separate or part of a larger software program for executing the method of 
Fig. 5. It is to be understood that the page layout subroutine 140 may be executed 
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by a variety of alternative methods and optimization techniques. The height and 
width of the n images to be arranged on the page and the format of the page are 
passed to the page layout subroutine 140. A trail page layout 212 is generated that 
contains all n images such that no two images overlap. 
5 In the embodiment illustrated, two useful methods that can each be 

used for generating non-overlapping placement of the images on a page, are the 
sequence pair and bounded sliceline grid BSG structures. The sequence pair 
structure is described in H. Murata, K. Fujiyoshi, S. Nakatake, and Y. Kajitani, 
"VLSI subroutine placement based on rectangle-packing by the sequence pair," 

10 IEEE Trans. Computer-Aided Design, vol. 15, pp. 1518-1524, Dec. 1996 and the 
BSG structure is described in S. Nakatake, K. Fujiyoshi, H. Murata, and Y. 
Kajitani, "Subroutine placement on BSG-structure and IC layout applications," 
Proc. IEEE Int. Conf. Computer-Aided Design, pp. 484-491, 1996. 

At step 200, the trail page layout 212 is generated by using one of 

15 the above methods to produce a non-overlapping placement of the n images on a 
plane and then isotropically scaling (i.e. the width and height are scaled by the 
same proportion) each image by the same proportion so as to fit onto the chosen 
page format. FIG. 8 illustrates a trail page layout 212 that contains the five 
images shown in FIG. 3. The trail page layout 212 is then scored by at step 210 by 

20 a cost (or objective) function. In the preferred embodiment the cost function is 
equal to the white space as defined by EQ. #1 . 

The goal of the optimization is to find a page layout that minimizes 
the cost function. In the preferred embodiment a simulated annealing approach is 
used to find an optimal page layout. For simulated annealing, a modification or 

25 change is made to the current trial page layout 212 to generate a new trial page 
layout 222 (see Fig. 9) at step 220. New trail page layout 222 is generated by 
randomly changing the relative positions of the images in the current trail page 
layout 212. The new trail page layout 222 is then scored at step 230 by the cost 
function. FIG. 9 illustrates the new trial page layout 222 that was generated by 

30 randomly perturbing the page layout shown in FIG. 8. From herein the prior trail 
page layout refers to the trail page layout that was modified to generate the next 
new trail page layout. 
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A determination is made as whether to keep the new trail page 
layout 222 or the prior trail page layout 212 at step 240. The new trial page layout 
222 is accepted if its score is lower than the score of the prior trial page layout 
212. Also, if the score for the new page layout 222 is greater than the score the 
5 prior page layout 212, the new page layout 222 is kept with a defined probability 
of exp[-(Ascore)/T] where Ascore = (score of new page layout - source of prior 
page layout). The parameter T is used to adjust the probability of keeping a new 
page layout. The optimization process is iterated at steps 220, 230, 240, and 250 
until the last iteration is obtained. The last iteration can be defined by the total 

10 number of iterations or by a number of iterations without an improvement. It is to 
be understood that any desired criteria may be used for determining when no 
further iteration is needed or desired. 

Optionally, further improvements in the calculated page layout can 
be made by further scaling the images by different amounts so to further minimize 

15 the white space. Once again a simulated annealing approach may be used. A new 
trail page layout is generated by randomly selecting an image and scaling it by a 
random factor between a minimum factor scalel and a maximum factor scale2 
while respecting the relative positions of the images imposed by the page layout 
calculated by iterating through steps 220, 230, 240 and 250. Scalel represents the 

20 lower scaling limit and scale2 represents the upper scaling limit. By varying the 
scale limits scalel and scald, page layouts can be generated that possess very 
different artistic look and feel. A small magnitude difference between the scaling 
factors scalel and scald (e.g. scalel = 0.9 and scale2 =1.1) ensures that no one 
image will overwhelm another image on the page. A large magnitude difference 

25 between scalel and scald can lead to optimized page layouts with very little 

white space where the images have very diverse sizes. A new page layout 272 is 
scored and compared to the prior page layout to determine which page layout to 
keep. Through an iterative process of modifying, scoring, and comparing a new 
page layout 272 such as shown in FIG. 10 is determined at step 260. In the new 

30 optimum page layout of Fig. 10 was generated by scaling each image in the page 
layout shown in FIG. 9 by random factors between 0.9 - 1 . 1 . 
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In a further optionally step 270, the images in the page layout can 
be aesthetically balanced. Aesthetic considerations play a role determining what 
makes a visually appealing page layout to an individual. There are many different 
artistic looks and feels that a page layout can possess and is very personal. In a 
5 preferred embodiment, the images are aesthetically balanced by positioning them 
on the page so that they are equally distributed on the page with visually appealing 
borders between each image and the border of the page while still preserving the 
amount of white space. For example, but not by way of limitation, the images are 
positioned on the page such that the left and right borders (space) 273, 274 

10 between an image and the next closest image or boundary (peripheral edge) of the 
page are made equal as shown in FIG. 12. The same is done for the top and 
bottom borders 275, 277 of the images as shown in FIG. 12. Note that the 
left/right 273, 274 and bottom/top 275, 277 borders of the images do not 
necessarily have to be equal. In addition, images that are close to the boundary of 

15 the page can be made left and/or right and/or top and/or bottom justified. 

Referring to FIG. 16, there is shown a flow chart of an embodiment 
for spatially balancing the images and equalizing the top/bottom and right/left 
borders of the images on a page according to the present invention. It is of course 
understood that the process of equalizing the borders/spaces around an image may 

20 be executed by a variety of alternative methods and optimization techniques. The 
process of generating equal top/bottom and equal left/right borders around each 
image in the page layout is an iterative process where the vertical and horizontal 
spacing between the image are determined independently. At step 300, the n 
images are sorted according to their vertical (y) axis 302 (see Fig. 1 1). Next at 

25 step 310 the bottommost image is set to the current image. The spaces between 
the closest image or page boundary above and below the current image are 
determined at step 320. At step 330, the current image is moved vertically so that 
the spaces (borders) between itself and the image or page boundary above and 
below are equal. At step 340, the closest image above the current image is set as 

30 the new current image. Steps 320, 330, 340 and 350 are repeated until the topmost 
image has been repositioned. The analogous process is repeated at steps 360, 370, 
380, 390, 400 and 410 for the horizontal spacing, as shown by arrow 304 in Fig. 
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11, between the images. The steps 300, 310, 320, 330, 340, 350, 360, 370, 380, 
390, 400, 410 and 420 are then iterated until the images have reached their 
equilibrium position. That occurs when none of images are repositioned through a 
single iteration of steps 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400 and 
5 410. At this point images are spatially balanced and the borders between the 
images have been equalized. 

FIG. 1 1 illustrates a page layout that was generated by equally 
distributing the images in the page layout 276 shown in FIG. 10. As shown in 
FIG. 1 1 some of the images are either touching each other and do not have white 

10 space between them or touching the boundary of the page. To ameliorate this 

look, the images can be scaled down and centered in their current locations. FIG. 
12 illustrates a page layout 278 where the images in FIG. 1 1 have been scaled 
down by a factor of 0.95 and centered in their current locations. 

To generate a more whimsical or scrapbook look the images can be 

15 randomly rotated or rotated in a predetermined pattern. The rotation of the images 
should be constraint such that the rotated images do not overlap or have a 
maximum overlap so as not to obscure important detail in the images. For 
example, no overlapping image which is greater than 10% or that avoids covering 
the central area of the image. FIG. 13 shows a page layout 280 where the images 

20 in page layout shown in FIG. 12 have been randomly rotated between -5 and +5. 

It is to be understood that various other modifications may be 
made. For example, but not by way of limitation, both the placement and scaling 
of the images can simultaneously be optimized in the calculation of the image 
page layout. The cost function can be generalized to contain many different goals 

25 in determining the page layout. Thus, in a further embodiment, the cost function 
can also take into account the aesthetic value in the calculation of the image page 
layout. In yet another embodiment of the invention, the optimization can 
minimize white space while simultaneously trying to avoid placing any images in 
a predefined location on the image page. FIG. 14 illustrates a page layout 282 

30 where the five images 54, 56, 58, 60, and 62 were placed so as not to be in the 
predefined location 283 represented by the cross hatched region. While in the 
embodiment illustrated by FIG. 14, the location 283 is in the lower right area. 
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However, location 283 may be anywhere on the page. For example, but not by 
way of limitation, location 283 may be in the center of the page with the images 
surrounding the location 283. This maybe especially useful when an image or 
artwork to be placed in the central location is related to the other images. In yet 
5 another embodiment, the location of a given image can be constrained to appear in 
a predefined location on the page layout. In yet another embodiment, the scaling 
factors used to scale the images can be constrained in such a way as to emphasis a 
given image relative to the other images. FIG. 15 illustrates a page layout 284 
containing the five images 54, 56, 58, 60, and 62 were the scaling factors were set 

10 so as to emphasis image 58 over images 54, 56, 60, and 62. 

Optimization techniques try to find the global minimum, for 
example, the image layout that has the lowest possible amount of white space, but 
they are susceptible to being trapped in a local minimum. Since the process of 
optimizing the page layout is a random process, different optimal page layouts can 

15 be generated each time the same images and page format are run through the page 
layout subroutine 140. The system can automatically iterate through the page 
layout subroutine generating a predefined number of image page layouts. The 
page layout that has the highest score is then chosen as the preferred page layout. 
Though it should be noted that the most aesthetically pleasing page layout might 

20 not correspond to the page layout that has the minimum amount of white space. 

The invention has been described in detail with particular reference 
to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 
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WHAT IS CLAIMED IS: 

1 . A method of organizing a plurality of images in a 
predetermined page format, comprising the steps of: 

grouping said plurality of images into a plurality of different page 

layouts; 

analyzing each of said different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on said predetermined criteria. 

2. A method according to claim 1 further comprising placing 
said plurality of images in said selected page layout. 

3. A method according to claim 1 wherein said predetermined 
criteria comprises the amount of white space in each of said page layouts. 

4 A method according to claim 3 wherein said analyzing said 
different page layouts comprises scoring each of said different page layouts. 

5. A method according to claim 1 further comprising the step 
of further scaling the images of said selected page layout by different amounts. 

6. A method according to claim 3 wherein the amount of 
white space is minimized by using stochastic algorithms. 

7. A method according to claim 1 wherein said predetermined 
criteria includes placing images in said different page layouts in a non- 
overlapping pattern. 



8. A method according to claim 1 wherein said placing of said 
plurality of images in said different page layouts comprises scaling all of said 
images such that they fit within said page format. 
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9. A method according to claim 1 wherein said analyzing of 
said different page layouts comprises a iteration of comparing sequentially two 
different page layouts and selecting the best page layout until little or no further 
improvement in scoring is obtained. 



10. A method according to claim 9 further comprising the step 
of scaling individual images of the page layout obtained after said iteration. 

11. A method according to claim 9 further comprising the step 
of rotating said images a predetermined amount. 

12. A method according to claim 1 further comprising the step 
of spatially balancing the spacing between said images. 

13. A method according to claim 1 further comprising the step 
of positioning said images in said selected page layout so as to provide a desired 
border on said page. 

14. A method according to claim 12 wherein said white space is 
determined vertically between adjacent images in said page layouts. 

15 A method according to claim 12 wherein said white space is 
determined horizontally between adjacent images in said page layouts. 

1 6. System for organizing a plurality of images in a 
predetermined format, comprising: 

a first computer for composing a plurality of digital image on a 

page; 

a software program such that when loaded on said computer will 
causes said computer to group said plurality of images into a plurality of different 
page layouts; 



-17- 

analyzing each of said different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on said predetermined criteria. 

17. A system according to claim 16 where in said computer can 
be accessed remotely over a communication network. 

1 8. A system according to claim 17 said computer is accessed 
by a second computer. 

19. A system according to claim 18 wherein said software 
program is run on said first computer. 

20. A system according to claim 18 wherein said second 
computer is personal computer of a customer. 

21 . A system according to claim 1 7 said computer is accessed 
by a retail kiosk. 

22. A computer software product for organizing a plurality of 
images in a predetermined format comprising a computer readable storage 
medium having a computer program which when loaded into a computer causes 
the computer to perform the following steps: 

grouping said plurality of images into a plurality of different page 

layouts; 

analyzing each of said different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on said predetermined criteria. 



23 . A method of organizing a plurality of images in a 
predetermined page format, comprising the steps of: 
providing a plurality of digital images; 
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selecting a number of said images for placement on said 
predetermined format; 

grouping said plurality of images into a plurality of different page 

layouts; 

analyzing each of said different page layouts in accordance with a 
predetermined criteria; and 

selecting the page layout based on said predetermined criteria. 
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ABSTRACT OF THE DISCLOSURE 

A system, software product and method for organizing a plurality 
of digital images in a predetermined page format. The digital images are grouped 
in a plurality of different page layouts. The page layouts are analyzed in 
accordance with a predetermined criteria. A page layout is selected based on the 
predetermined criteria. 
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